Skip to content

Comments

From NVIDIA Megatron-LM for visibility#18

Open
RaymondLi0 wants to merge 6335 commits intobigcode-project:multi-query-attentionfrom
NVIDIA:main
Open

From NVIDIA Megatron-LM for visibility#18
RaymondLi0 wants to merge 6335 commits intobigcode-project:multi-query-attentionfrom
NVIDIA:main

Conversation

@RaymondLi0
Copy link
Collaborator

@RaymondLi0 RaymondLi0 commented Jan 24, 2023

No description provided.

@RaymondLi0 RaymondLi0 changed the base branch from multi-query-attention to before-merge June 20, 2023 20:12
@RaymondLi0 RaymondLi0 changed the base branch from before-merge to multi-query-attention June 20, 2023 20:12
hexinw-nvidia and others added 28 commits January 26, 2026 16:19
@hexinw-nvidia
Added --ft-num-warmup-iters option. (#3052)
06d0f46
@jiemingz
Reapply "Various CUDA graph improvements on capture time, replay time…
642fdd9 
…, memory footprint (#2572)" (#3056)

Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com>
@liuyun7345 @shjwudp
fix(fsdp): add CLI argument for outer_dp_sharding_strategy (#3053)
bb42a00 
Co-authored-by: Jianbin Chang <shjwudp@gmail.com>
@ko3n1g
ci: Log node name (#3081)
94d8186 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@jeffnvidia
add all_gather process-group for overlapping in fsdp disributed train…
528cb2e 
…ing (#2663)
@ko3n1g
docs: Release docs (#3055)
23a76d1 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ChenhanYu @jenchen13 @AAnoosheh
Support NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 FP8/NVFP4 PTQ in example (#…
b47c376 
…3079)

Signed-off-by: jenchen13 <jennifchen@nvidia.com>
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
Co-authored-by: jenchen13 <jennifchen@nvidia.com>
Co-authored-by: Jenny Chen <jc4686@columbia.edu>
Co-authored-by: Asha Anoosheh <ashaanoosheh1@gmail.com>
@ko3n1g
Reapply "Support multimodule communication (#2031)" (#3068)
35e85a6 
This reverts commit 0972f02.
@litianjian @ISEEKYAN
@Phlip79 @sidsingh-nvidia
Add router replay for MoE models (#2101)
db6b895 
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: Yan Bai <baiyan1996@icloud.com>
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
Co-authored-by: Siddharth Singh <136645615+sidsingh-nvidia@users.noreply.github.com>
@ko3n1g
ci: Disable gpt_dynamic_inference_tp1_pp1_dp8_583m_throughputtest_zmq (…
7031953 
…#3099)

Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
ci: Repeat func tests, save logs of unit tests and lessen debug output (
dea21a0 
#3089)

Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
ci: Update improvement of step-time (#3104)
dd83fc6 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
ci: Add GPU health checks (#3100)
2bdf7e1 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@jon-barker
Harden GRPO functional tests (#3065)
d68721b 
Co-authored-by: Jon Barker <jbarker@oci-hsg-cs-001-vscode-01.cm.cluster>
@mathemakitten
Inference functional tests: Write outputs to INFERENCE_OUTPUT_PATH in…
4015ff1 
…stead of TENSORBOARD_PATH (#3061)
@Victarry
Update moe readme. (#2830)
0888a06
@ko3n1g
build: Bump to TE2.12 (#3086)
2b02a28 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@deepakn94
Logging cleanup (only log on rank 0 if possible) (#3036)
6cf285b 
Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>
@Phlip79
Move all bert and t5 tests to nightly (#3106)
65217aa
@Phlip79
Create greptile.json (#3087)
4fb549f
@github-actions
Update copy-pr-bot.yaml [skip ci]
6273d74
@kunlunl
Fix bug of reuse_grad_buf_for_mxfp8_param_ag (#2802)
33224cc 
Signed-off-by: kunlunl <kunlunl@nvidia.com>
@parthmannan
Fix for Hybrid CP (#3091)
fb6a592
@jon-barker
Fix GRPO re-fit functional test (#3113)
f6c8a61
@megnvidia @Phlip79
Minimize README contents (#3020)
991138e 
Signed-off-by: meg miranda <mmiranda@nvidia.com>
Co-authored-by: Philip Petrakian <pgpetrak@gmail.com>
@shjwudp
Add end-to-end tests for M-FSDP and ND-Parallel (#3031)
964c902
@faradawn @maanug-nv
Fix Multimodal Dockerfile (#3006)
38cd9fc 
Co-authored-by: Maanu Grover <109391026+maanug-nv@users.noreply.github.com>
@shjwudp @cspades
[M-FSDP] Fix double buffering not working with activation recompute (#…
b6b49e7 
…2689)

Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>
yobibyte and others added 29 commits February 18, 2026 14:40
@yobibyte
Log RL metrics per environment (#3446)
f1908bc
@tdene @mathemakitten
Move tensor offload/onload out of RL code (#3029)
1106df4 
Co-authored-by: helen ngo <helen.ngo14@gmail.com>
@ArEsKay3
Add Engine event to the follow up requests after checkpointing (#3473)
0672477
@tdene @ArEsKay3
Fix another inference flask / Gym interaction (#3467)
7b016be 
Co-authored-by: Robert Kirby <rkirby@nvidia.com>
@megnvidia
adding in copyright blurb at the top of md file (#3394)
acb7273 
Signed-off-by: meg miranda <mmiranda@nvidia.com>
@shjwudp
[Megatron-FSDP] Add fsdp_all_gather_in_start_param_sync option in DDP…
fdde15e 
… Config (#3095)
@chtruong814
ci: Update release workflow to include changelog and publish docs (#3472
77f22f2 
)

Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@ko3n1g
ci(fix): Weekly GPT tests (#3443)
1666b45 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
ci: Remove environments (#3462)
0d0943c 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@dimapihtar
update HF tokenizer defaults (#3440)
d07f16c 
Signed-off-by: dimapihtar <dpihtar@gmail.com>
@AAnoosheh @jenchen13
PTQ changes for upcoming QAD (#3124)
1d694c2 
Signed-off-by: Asha Anoosheh <aanoosheh@nvidia.com>
Co-authored-by: Jenny Chen <jennifchen@nvidia.com>
@ko3n1g
ci: Bump preflight to detect our svc (#3494)
655cc8e 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
build: Drop Python 3.10 support and pip install one-logger (#3485)
7f35af4 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
ci: Bump pre-flight for Bot SSO (#3497)
2d06cc9 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
a781f3c
@ko3n1g
Revert "build: Drop Python 3.10 support and pip install one-logger (#… (
50ebe8e 
#3500)
@santhnm2
Fix chunked prefill edge cases (#3404)
c191ae8 
Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>
@ko3n1g
ci: Enable MBridge downstream testing via PR (#3483)
9f611b7 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@chtruong814
ci: Remove gitlab docs build job and set LTS integration and function…
7b6e226 
…al tests to allow failure (#3349)

Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@yueshen2016
[OMNIML-3232] ModelOpt: add full TE spec option and wire Mamba stack …
31bd4a3 
…/ post-training scripts (#3393)
@tdene
Track off-policyness across RL steps (#3030)
9ba248e
@github-actions
chore(beep boop 🤖): Bump (main) (2026-02-20)
9d72f63
@ko3n1g
ci: MBridge testing branch name during merge-queues (#3513)
b7aa6a0 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g @claude
ci: Enable Dependabot Automerge (#3487)
7a36263 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
@ko3n1g
ci: Also sync direct teams (#3484)
e8fd432 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@faradawn
Multimodal: fix argument checking (#3449)
01b361c 
Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
@cspades
Fix Megatron-FSDP optimizer state DCP checkpointing, and fix DTensor …
773c113 
…deepcopy bug from PyTorch 26.01. (#3510)

Signed-off-by: Cory Ye <cye@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
b555baf
@sidsingh-nvidia
Renable full_iteration cuda graphs for inference. Add them for the ma…
32efeff 
…mba block. (#3250)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.